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New Capability: Developer Desktop 


Preconfigured Virtual Desktop for Software Development 
o No need to take time to setup developer environment 
e Software development applications are already loaded 
e Jetbrains IDE, Eclipse, GitHub/GitLab, Visual Studio, Al Plugins 
Accessible from a Browser 
>» No special client downloads 


Entirely OpenSource 
o No restrictions due to licensing 


Multiple O/S workstations available 
> Centos7 initially; Other Linux & Windows later 


Platform 
o Instance-based initially 
o Container-based (AKA Kubernetes) later for applications not requiring a full O/S 


Developers find same/similar environment on various projects/networks 
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Simple Concept: Virtual Developer Workstation 


Requirements Solution 
e Centos7 based virtual workstation e Build acentos7 Instance 
o.” Accessible from available windows/linux clients o.” Preconfigured w/things a “developer” needs 
a Use RDP wrapped in TLS a Eclipse, Editors, Desktop 
o  Preconfigured with “typical” developer apps = Development environment (Java, C++, 
wi Eclipse, compilers, editors, Docker, et. al etc. 
= Meets security requirements a Use Ansible to configure workstation 
ë Multi-Cloud “capable” o Set of tools preconfigured to meet security 
ə Give developers “something” requirements 
o That is easy to use - preconfigured o Save developer configuration time 
o Saves them time from having to configure a e Access from a web browser 
workstation themselves o Use HTML5 (Most Web clients) 
o Reasonably secure o Apache Guacamole 
= But doesn’t stop them from configuring o Considered RDP client with TLS tunnel, but 
their unique development environment didn’t meet audit requirements 


Scale to hundreds of users 
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Really want to avoid 


OK. I've got this great Open Source product. I've tested it and it works 
great. What do | do to provide this awesome functionality to my 
hundreds of users. 


e All too typical: 


O 


O 


O 


Q 


Load onto an accessible instance 
Update instance size when users complain of slowness 
Reboot if problems 


Spend inordinate amount of time with users; changing passwords, 
rebooting, general support, updating public/private keys, etc. 


“Been there; Done that” 
Security is an afterthought 
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Talk's Focus 


Guacamole 

Server - , Developer 

HTTP (Docker Desktop 
Container) 
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Postgres 
Database 
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Developer 
Desktop 


Proof-of-Concept 


Other AWS Services: 
Cloudwatch: Statistics, Alerts & Alarms 
AWS Secrets 


a Production 


Cloudformation 
AWS RDS 


Maintainable: secure, key rotation, updates, new versions 
AWS focused now but vendor neutral growth 


Load balancer does a health check and if failure then 
Autoscale will in turn fail and regenerate the instance 


AWS RDS will automatically backup and update the 
Postgres database 


Use a Commercial Product ? Possible But... 


e Typical Issues 
o Workstation not general enough of customization not sufficient 
o Not secure enough 
o Needs custom client download 
o License fees 
o Lack of control 
o Not multi-cloud 
o Proprietary 


e Our Needs 
o We want to create a paradigm to support ease of use for the developer 
o Want flexibility 
o Not force a paradigm on the developer 
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Guacamole: 
Aside from a really yummy dip for chips... 


e Guacamole is: e Started by Michael Jumper and James Muehlner, 


o A protocol circa 2010 
o AcClientless Remote Desktop Gateway 


a  Clientless: Only a web browser required e Initially developed on SourceForge 
- no plugins, no software installs e Entered Apache Software Foundation (ASF) 
= Remote Desktop: Support for common “Incubator” project in 2016 


remote desktop protocols 


a. aaa e Graduated to ASF Top Level Project (TLP) in late 


e RDP 2017 

e SSH e Licensed under the Apache 2.0 License 
e Telnet 

e VNC 


a Gateway: Web-based, authentication, 
and permission control 


Slide contents provided by Nick Couchman/Apache Guacamole 
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Configuration: Understand configuration options 


Traceability 
o Code 
o Glue code 
o Logging 


Cost Control 
Maintainability 
o  Public/private keys; UserlDs/passwords 
o.” Administration 
= Infrastructure 
a Users 
= Support 
OpenSource Software is “Ready to Use” 
o All project CM done by the opensource project 
o Only configuration file(s) need to be generated 
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What “Production” means (to me) 


OpenSource project provides function “Buffet” to meet production needs (Oauth, OpenID, SAML, etc.) 


Add “stuff” not supplied by the project repo 
o Site Specific Information & Localization 
=» Branding 
= Security needs 
= Infrastructure 
Other Considerations 
o Updates 
a Project software as needed 
a Other software 
e Scripts 
ə Downloaded applications 
o Security in general 
o Keep project software load “clean”; no changes 
o  UserlDs/passwords, public/private keys secure 
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Utilize AWS for Production Environment Integrations 


e Log File Histories: 
AWS Cloudwatch 8: AWS 53 


e Statistics: 
AWS Cloudwatch 


e Alerting (Alarms & Notifications) 
AWS Cloudwatch 


e Automation: Building & Management 
o CloudFormation 
AutoScaling 
Load Balancer 
Other AWS Infrastructure: Security groups, IAM Policy, VPC/subnets, AMI, instances 


e Secrets 
>» AWS Secrets 
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Additional Needs for Production (1/3) 


e Guacamole needs a datastore to store user & connection data 
. Default is a data file for connection parameters and UserlDs/passwords 
e Other methods supported: we use AWS RDS (Postgres) and OpenID 
e Issues Management: 
o Git 
o Software Development Management: Jira 
e Manage & store public/private keys: 
o Update & Rotate Keys 
o Includes signed public keys not generally in keystores 
o Key Stored in AWS Secrets 
e Managed access via AWS IAM policies 
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Additional Needs for Production (2/3) 


e Congestion Handling 
o AWS Autoscaling 
o AWS Load Balancer 


e Create base AMIs 
. Currently: run scripts manually 
o Planned: Create monthly via AWS Lambda 


e Configuration Management 
“Vanilla” repository 
e Scripts, templates, infrastructure (Cloudformation) 
o Site Specific Repository 
e Location Unique identifiers repository (yaml/json files) 


e Blob Repository 


o Nexus 
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Additional Needs for Production (3/4) 


e Location Unique Credentials: 
o AWS Secrets 
o Scripts to generate/access these Credentials 
o Infrastructure parameters (VPC, Subnet, AMI IDs, IAM Policy. etc.) 


e Log File Histories; Statistics; Alerting (Alarms & Notifications) 
o AWS Cloudwatch & AWS S3 
o Use the unified CloudWatch agent 


e Automation 
o Building 
e Should be scripted; parameters in yami file 
o Management 
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Additional Needs for Production (4/4) 


e General Security 


e Architecture: 
o Isolated with security groups in separate zones/subnets 
e Developer Desktop instances 
e Guacamole 8 Database 
e Load balancer (Accepts only HTTPS (port 443)) 


e Isolation via subnets; AWS security groups 
e Encrypted data at rest 


e Encryption in transit TLS Encryption: 


Guacamole and Guacd 
User and load balancer 


pa AWS S M S Load balancer and Guacamole 
Guacd and User’s Desktop Instance 
o Periodic updates; Periodic security scans Guacamole and database 


All AWS services connections 
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Feature Build Breakdown 


e Build custom AMIs 
o Create DD AMI w/ needed software for Developer Desktop Instance 
o Create Guacamole AMI w/Guacamole &Guacd 
o Use Hashi's Packer w/ Ansible 


o Template file driven python scripts 
e For DD EC2 deployment 
e For CloudFormation Guacamole deployment 
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Secrets & Management 


Secrets 
Secrets Management 


e 6 public/private key pairs to rotate (for TLS) 
o Load Balancer <-> user 
o Tomcat <-> Load Balancer 
o Servlet (guacamole.war) <-> Guacd 


e Scripts to push keys into AWS Secrets 
o Run as needed 
o Can create self-signed or use signed keys as 


o Servlet (guacamole.war) <-> Database (SQL) needed 
o  Guacd <-> User's Instance (RDP) e Public keys added to keystore during AMI build 
o OpenID server <-> Servlet (Guacamole.war) (Packer/Ansible) 
e Database UserlD/password e Private keys read from AWS Secrets using 
e Other tokens/public keys Cloud-Init during instance deploy 
o Nexus Blob Repository token (needed during e Refresh keys done by 
AMI build) o Pushing keys to AWS Secrets 
o AWS public keys o Rebuilding AMI 
e User Login to Guacamole managed with =  W/public keys 
OpenID server o.” Re-deploying using AWS Autoscaler 
e Desktop User Logins/Passwords also stored by = Private keys added 
Guacamole 
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Encryption Key Updates 


e Public key updates need new AMI 
o Update keys in AWS Secrets 
o Rebuild AMI 


e Private key updates need new deploy 
o Update keys in AWS Secrets 
o Redeploy (rolling update) 


e Allows existing connections to drain 
e New connections are routed to new instance 
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Keys/Passwords Scripts: Calls & Returns 


e Wrote script to either read keys from files or generate a self-signed key 
pairs and load keypairs into AWS Secrets 


e ./gen-guac-ssk 
o -C <true if create self-signed keys | false to read keys from files> 
o -p< true to push keys and passwords to AWS Secrets manager | false to export 
keys and passwords to env> 
e To rotate keys: 
o Run: source ./gen-guac-ssk -c true -p true 


o Then use AWS Autoscaling to terminate current Guacamole instance and deploy 
new 
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AMI Generation 


e Packer scripts with Ansible plugin 
o Site specific variables defined in Packer HCL yani file 
o Site specific variables defined in Ansible Vars directory yaml file 
o Parameters (source AMI Id, et. al.) in bash wrapper script 


e Separate but similar scripts for Developer Desktop AMI & Guacamole 
(Guacamole & Guad) AMI 
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AMI Scripts: Calls 8: Returns 


e Developer Desktop AMI 


more dd-ami 


#!/usr/bin/bash 

if [[ "xS1" == "x" ]]; then echo "var file not present -exiting"; exit 1, fi 
echo "using \"$1\" as packer var file" 

read -p "Are you sure? " -n 1 -r 

if [[ ! SREPLY =~ ^[Yy]$ ]]: then echo -e "\nresponse must be \"Y\" or \"y\" to continue. Exiting"; exit 2; fi 
echo 

#create new version number 

MAJOR="0" 

MINOR=S(($(cat DD_version)+1)) 

echo SMINOR >DD_version 

PAD="" 

if [[ MINOR -It 10 ]]; then PAD="0"; fi 

if [[ MINOR -It 100 ]]: then PAD="0SPAD"; fi 
DD_version="VS{MAJOR}.S{PAD}${MINOR}" 

echo "NEW AMI Version: SDD_version" 

#Run Pack 

#packer validate -var "ami_force_deregister=true" ami 

#get certs for DD nexus repo 

. ./get_certs 

export DD_ADMIN='DEFAULT_USER' 

packer build -var-file "$1" -var "dev wkstn version-SDD version" ami.dd 


e Guacamole AMI 


more dguac-ami 


#!/usr/bin/bash 

if [["x$1" == "x" ]]: then echo "var file not present -exiting"; exit 1; fi 
echo "using \"$1\" as packer var file" 

read -p "Are you sure? " -n 1 -r 

if [[ ! SREPLY =~ ^[Yy]$ ]]; then echo -e "\nresponse must be \"Y\" or \"y\" to continue. Exiting"; exit 2, fi 
echo 

#create new version number 

MAJOR="0" 

MINOR=S(($(cat Guac_ami_version)+1)) 

echo SMINOR >Guac_ami_version 

PAD="" 

if [[ MINOR -It 10 ]], then PAD="0"; fi 

if [[ MINOR -It 100 ]]; then PAD-"OSPAD": fi 
Guac_ami_version="VS{MAJOR}.S{PAD}S{MINOR}" 

echo "NEW Guac AMI Version: SGuac ami version" 

#Run Packer 

#packer validate -var "ami force deregisterz-true" ami 

#get certs for DD nexus repo 

pushd .. && . ./get_certs && popd 

export DD_ADMIN='DEFAULT_USER' 

packer build -var-file "$1" -var "guac_server_version=SGuac_ami_version" ami.guac 


Note: var file contains files of variables in YAML format 


YAML files provide configuration management 
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User Desktop Deployment 


e Bash script with call to generalized Python script EC2 deployment 
o Generates the User's instance thru calls to EC2 API (BOTO3) 
o Updates Guacamole database 
e Adds user data 
e Adds associated instance connection data 
o EC2 Deployment is template driven 


e Details: 
o Script arguments include instance UserID, Password, public key 
o Reads a template yaml file containing subnet ID, AMI ID, userdata, etc. 


o Creates yami file with all particulars for visual verification and CM; also logs deploy 
data for historical record. 
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User Desktop Deployment: Calls 


wrapper script for a python program that deploys an 


EC2 


o Creates a user's DD instance and updates the guacamole 


database: 


./dd-deploy -u user -p password -k "ssh-rsa AAAA...." -a "ami-0000000000000" 


YAML files provide configuration management 
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Example parameter file for runec2: 


more DD deploy parameters.yaml.readme 
REGION : 'us-east-1' 
EC2-OPERATION : 'create' 
EC2-PARAMETERS : 
- SecurityGrouplds : 
- 'sg-0000000' 
- Subnetld : 'subnet-0000000000' 
- InstanceType : 't3.medium' 
- KeyName : 'my-pem' 
- Imageld :" 
- MaxCount : 1 
- MinCount : 1 
- TagSpecifications : 
- ResourceType : 'instance' 
Tags: 
- Key : 'Name' 
Value : 'DD-' 
- Key :'ID' 
Value : '00001' 
- UserData : | 
#!/usr/bin/bash 
# 
# 


#setup named user 
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Guacamole Capacity 


e AWS Load balancer scales as needed to handle users volume 


e Autoscaling used to create more guacamole instances as needed 
o Average CPU usage > 60% triggers autoscaling up 
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Guacamole Deployment 


e Bash script with call to generalized Python script Cloudformation 
deployment 


o Specifies the Guacamole instance thru autocale template init and userdata 
Instance 


o Setup up statistics collection, alarms and TBD 


o Parameter driven using yaml/json file to provide site parameters 
e VPCID, subnet ID, AMI ID, autoscale parameters, IAM policy ID 


e Details: 


o Script arguments include CloudFormation update, create, delete. 


o Reads a parameter file containing VPC ID, subnet ID, AMI ID, autoscale parameters, 
IAM policy ID, etc. 


o Produces log file for record 
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Guacamole Deploy: Calls & Returns 


e Deploy cloudformation template: 
python runstk.py --log-level DEBUG -pf parameters-guac-server.yaml 
Sample parameters-guac-server.yaml: 


Region : 'us-east-1' 

StackName : 'My Stack’ 

DisableRollback : 'True’ 

CAPABILITIES : 'CAPABILITY_IAM’ 

TEMPLATE-FILE : 'my_cfn.yaml’ 

TEMPLATE-OPERATION : 'create’ 

STACK-PARAMETERS : 

- VPCld : 'vpc-O00000000’ 

- VPCldCidr : "10.10.10.0/24” 

- SubnetID : "subnet-0000000000000” 

- SecurityGroups : 'sg-000000000000, sg-000000000001, sg-0000000000002" 
- SystemsManagerAccess : 'false’ 

- KeyName : "my-pem“ 

- RedirectURL : 'internal-dev-00000000000.us-east-1.elb.amazonaws.com' 
- RedirectURLPort : ” 

- InstanceType : "t3.med" 
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Planned 


e Currently all scripts return error codes upon exit. But more is needed 
o Functionality tests needed to determine validity 

e DD AMI; Guacamole AMI 

e DD instance after deployment 

e Guacamole instance after deployment 
e Currently LB does a simple HTTPS ping but doesn’t test functionality 
e Test Guacamole/Tomcat functionality 
e Test Guacd functionality 


e Other Developer instances 
o Other versions of Linux instances; windows versions 
o Other versions of K8s O/S instances 
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Next Generation Architecture 
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Developers Desktop: 
Centos7 
Centos8 
Centos9 

Other Linux O/S 
Any Windows 


Virtual or containers 


Amazon Web Services 


Dunannmalan | 
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Developing Infrastructure for Using Guacamole 


Thank You 
Questions? 
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